Package net.paulhertz.pixelaudio

Class PixelAudioMapper

java.lang.Object
net.paulhertz.pixelaudio.PixelAudioMapper
public class PixelAudioMapper extends Object

As of pre-release version 0.9.2-beta, PixelAudioMapper is substantially complete, though there are a number of features that have not been tested or demonstrated with code examples.

PixelAudioMapper maps between 1D "signal" arrays of audio samples formatted as floating point values in the range [-1, 1] and 2D "image" arrays formatted as RGBA integer pixel data. This class is designed to handle one-to-one mappings between signal and image arrays. The mappings are managed by lookup tables (LUTs) created by a separate mapping generator class, PixelMapGen. The values in the LUTs are index numbers of pixels in a bitmap or of samples in a signal. If you think of the signal as a path that visits each pixel in the image, one lookup table, signalToImageLUT, lists the index numbers of each pixel the path visits in the image, in the order that it traverses them. There is a similar lookup table for the image, imageToSignalLUT, that lets you look up the signal value corresponding to each pixel in the image. For example, when you load audio samples to the signal array, signalToImageLUT lets you find the corresponding pixels for each sample and update them to visualize the signal as a 2D image. You can save the image to a file and later load it to a bitmap for display. The pixel values can then be written to an audio buffer using imageToSignalLUT.

Some typical uses for this class include:
  • Reading an audio file or audio stream into the signal array and then writing its values, transcoded to RGB integer values, to the image array for display as a visualization.
  • Using interaction with an image to trigger audio events at precise locations in a signal.
  • Running audio filters on an image-as-signal and writing the results to the image.
  • Running image algorithms on a signal-as-image and writing the results back to the signal.
  • Synthesizing image data and audio and then animating the data while interactively triggering audio events.

DATA REPRESENTATION

PixelAudioMapper requires image arrays to contain standard 24- or 32-bit RGB or RGBA pixel data, in row major order, with (0,0) at upper left corner. It requires signal arrays to contain values in the range [-1.0, 1.0], a standard format for audio samples.

For the sake of generality, the enclosing classes for image and audio data remain external to PixelAudioMapper, which just works with the arrays of audio samples or image pixel data that they provide. In Processing, PImage wraps image data. You could also use Java's BufferedImage class. I have been using the minim library for audio, (https://code.compartmental.net/minim/). The built-in audio in Processing 4 is definitely also an option. PImage and BufferedImage typically store color pixel data in an array of RGB or RGBA integer formatted values -- exactly what we need. Audio classes use a variety of formats, particularly when reading from files, and provide methods for setting and changing the format of audio sample data.

Image

         Width w, Height h
         Index values {0..(w * h - 1)} point into the pixel array.
         Index to coordinate conversion for row major order with index i, width w, height h:
                i = x + w * y;
                x = i % w; y = i/w   // using integer math;
         Default data format: 24-bit RGB or 32-bit RGBA, for display from a bitmap to a computer monitor.
         RGBA includes an alpha channel A.

Signal

         Array with same cardinality as image data array {0..(w * h - 1)}
         Default data format: floating point values in the range  [-1.0, 1.0]

LOOKUP TABLES

At their most general, lookup tables or LUTs set up a one-to-one correspondence between two arrays of the same cardinality, independent of the format of their data values. Every element in one array corresponds to exactly one element in the other array. Starting from array A, for an element at index A[i] we find the index of the corresponding element in array B at aToBLUT[i]. An element j in array B has the index of its counterpart in array A at bToALUT[j].

In PixelAudioMapper, we employ two LUTs, signalToImageLUT and imageToSignalLUT, to map elements in signal or image to the corresponding position in image or signal.

        signalToImageLUT: integer values over {0..(h * w - 1)} map a signal array index to a pixel array index
        imageToSignalLUT: integer values over (0..(h * w - 1)} map an image array index to a signal array index

In signalToImageLUT, we can get the pixel index in the image for any index in the signal. In imageToSignalLUT, we can get index in the signal for any pixel index in the image.

Each array is the inverse of the other: for an array index i: 
        signalToImageLUT[imageToSignalLUT[i]] == i;
        imageToSignalLUT[signalToImageLUT[i]] == i;

Image data is always in row major order for PImage, our image data class. Signal values can be mapped to locations in the image in any arbitrary order, as long their coordinates traverse the entire image. A typical reordering might be a zigzag from upper left to lower right of an image, or a space-filling fractal, or even a randomly shuffled order. The coordinates of each pixel in the image are stored as indices (i = x + w * y) in signalToImageLUT.

Once we know the “pixel index” for each value in the signal and have initialized signalToImageLUT, we can initialize imageToSignalLUT:

        for (int i = 0; i < w * h - 1; i++) {
                imageToSignalLUT[signalToImageLUT[i]] = i;
        }

The LUTs are generated by a subclass of PixelMapGen that is passed as an argument to the PixelAudioMapper constructor. Each PixelMapGen subclass generates: 1. a set of coordinates for the path traced by the signal over the image, 2. pixelMap for mapping from signal to image (signalToImageLUT in PixelAudioMapper), and 3. sampleMap (imageToSignalLUT in PixelAudioMapper), for mapping from image to signal PixelAudioMapper works with copies of the two LUTs, and can access or obtain a copy of the coordinates if needed. This strategy allows the generator classes to be compact and reusable, while the host class, PixelAudioMapper, can handle exchanges between audio and pixel data using its copies of the LUTs. Note the the pixel array and the signal array length must equal the image size = width * height.

To work with PixelAudioMapper, first create a PixMapGen instance with the width and height of the image you are addressing. The PixMapGen instance will generate the LUTs for its particular mapping for you. You can then pass it to the PixelAudioMapper constructor, which will initialize its variables from copies of the PixMapGen LUTs. Some of the logic behind this process is explained in my notes to the PixMapGen abstract class.

MAPPING AND TRANSCODING

We typically use the LUTs whenever we change the data in the signal or the image and want to write the new values to its counterpart, updating the appearance of the image or the sound of the audio signal. If the values in the arrays are in different formats, we will need to transcode the values from one format to the other. We have two methods, in pseudocode here:

        mapSigToImg             map signal values to the image: img[i] = transcode(sig[imgLUT[i]]);
        mapImgToSig             map image values to the signal: sig[i] = transcode(img[sigLUT[i]]);

The img variable in the pseudocode corresponds to an array of RGB data from a bitmap class. The sig variable corresponds to an array of floating point samples from an audio class.

In addition, we can write image or signal values directly, without using the LUTs. This operation transforms the order of the pixel or signal values.

        writeImgToSig   write image values directly to the signal: sig[i] = transcode(img[i]);
        writeSigToImg   write signal values directly to the image: img[i] = transcode(sig[i]);

READING AND WRITING SUBARRAYS

// TODO rewrite this section

When we want to work with subarrays of data from the signal or the image, it can be ordered either by the signal or image array order or by mapping with the corresponding LUT. In the case of images, we also have standard methods of reading and writing rectangular selections. We can define some methods to read and write data either in the order determined by the signal or by rectangular areas in the image. We’ll call the signal order methods pluck (read) and plant (write), and the pixel order methods peel (read) and stamp (write).

Arguments to mapping and writing methods are written so that source precedes target. Using this convention, most methods have a unique signature that also indicates how they function. Where there are ambiguities or a need for clarification, I have renamed the function, as in pluckPixelsAsFloat, pluckSamplesAsInt, peelPixelsAsFloat, and peelSamplesAsInt.

ARRAY SHIFTING

Standard operations we can perform with the signal array:

   shiftLeft()          an array rotation where index values decrease and wrap around at the beginning
   shiftRight()         an array rotation where index values increase and wrap around at the end

Shifting has proved so useful for animation that I am including it in the class. The shift methods also demonstrate how to update the signal and pixel arrays. The WaveSynth class provides other methods for animation.

OTHER OPERATIONS

The following are suggestions for methods that could be implemented using PixelArrayMapper.

  • additive audio synthesis + color organ, implemented with the WaveSynth and WaveData classes
  • granular synthesis (AriaDemoApp is not quite GS, but very similar)
  • pattern generation (Argosy and Lindenmeyer classes)
  • phase shifting, amplitude modulation, etc.
  • FFT operations on both image and signal data (AriaDemoApp)
  • pixel sorting, typically on image data
  • blur, sharpen, etc.
  • blending images
  • mixing signals

UPDATING AUDIO AND IMAGE

As a rule, operations on the signal should be followed by writing to the image, and operations on the image should be followed by writing to the signal. This will keep the values synchronized, even though they have different numerical formats.

In most of the examples that accompany this library, audio data uses the Lightness channel of an HSL representation of the image's RGB data, but this is by no means the only way of doing things. Using the Lightness channel restricts audio data to 8 bits, apt for glitch esthetics, but noisy. It's also possible to maintain high resolution data in the signal by processing image and audio data separately, and writing audio data to the image but not in the other direction.

Finally, it bears mentioning that the image can be treated as simply an interface into an audio buffer, where events such as mouse clicks or drawing and animation trigger audio events but do not modify the audio buffer. Library examples will provide some suggestions for this strategy.

Note: Representing ranges

I am following the convention from mathematical notation of representing closed ranges with [] and open ranges with (). I occasionally lapse into the alternate notations {0..(h * w - 1)} or [0, 255]. When values are integers, it should be understood that the range covers only integer values. Floating point values are continuous, to the limits of digital computation.

See Also:

  • Nested Class Summary

    Nested Classes
    Modifier and Type
    Class
    Description
    static enum 
    PixelAudioMapper.ChannelNames
    List of available color channels, "L" for lightness, since "B" for brightness is taken.

  • Field Summary

    Fields
    Modifier and Type
    Field
    Description
    protected PixelMapGen
    generator
    PixelMapGenINF instance to generate LUTs
    protected int
    height
    image height
    protected int[]
    imageToSignalLUT
    Lookup table to go from the image to the signal: index values over {0..(h * w - 1)} point to a corresponding index position in the signal array sig[]
    protected int
    mapSize
    pixel array and signal array length, equal to w * h
    protected int[]
    signalToImageLUT
    Lookup table to go from the signal to the image: index values over {0..(h * w - 1)} point to a corresponding index position in the image array img.pixels[]
    protected int
    width
    image width

  • Constructor Summary

    Constructors
    Constructor
    Description
    PixelAudioMapper(PixelMapGen gen)
    Basic constructor for PixelAudio, sets up all variables.

  • Method Summary

    Modifier and Type
    Method
    Description
    static final int
    alphaComponent(int argb)
    Returns alpha channel value of a color.
    static int
    applyAll(float sample, int rgb)
     
    static int
    applyAll(int rgbSource, int rgb)
     
    static int
    applyAlpha(float sample, int rgb)
     
    static final int
    applyAlpha(int rgba, int rgb)
    Takes the alpha channel from one color, rgba, and applies it to another color, rgb.
    static int
    applyAudioToColor(float sample, int rgb, PixelAudioMapper.ChannelNames chan, float[] hsbPixel)
    Helper method that applies a float audio sample to a pixel channel, using the channel name.
    static int
    applyBlue(float sample, int rgb)
     
    static int
    applyBlue(int rgbSource, int rgb)
     
    static int
    applyBrightness(float sample, int rgb)
     
    static int
    applyBrightness(float sample, int rgb, float[] hsbPixel)
     
    static int
    applyBrightness(int rgbSource, int rgb)
     
    static int
    applyBrightness(int rgbSource, int rgb, float[] hsbPixel)
     
    static int
    applyChannelToColor(int rgbSource, int rgbTarget, PixelAudioMapper.ChannelNames chan, float[] hsbPixel)
    Helper method for applying a color channel from a to an RGB pixel.
    static int[]
    applyColor(int[] colorSource, int[] graySource, int[] lut)
    Utility method for applying hue and saturation values from a source array of RGB values to the brightness values in a target array of RGB values, using a lookup table to redirect indexing.
    static int
    applyColor(int colorSource, int graySource)
     
    static int
    applyColor(int colorSource, int graySource, float[] hsbPixel)
     
    static int
    applyGreen(float sample, int rgb)
     
    static int
    applyGreen(int rgbSource, int rgb)
     
    static int
    applyHue(float sample, int rgb)
     
    static int
    applyHue(float sample, int rgb, float[] hsbPixel)
     
    static int
    applyHue(int rgbSource, int rgb)
     
    static int
    applyHue(int rgbSource, int rgb, float[] hsbPixel)
     
    static int
    applyRed(float sample, int rgb)
     
    static int
    applyRed(int rgbSource, int rgb)
     
    static int
    applySaturation(float sample, int rgb)
     
    static int
    applySaturation(float sample, int rgb, float[] hsbPixel)
     
    static int
    applySaturation(int rgbSource, int rgb)
     
    static int
    applySaturation(int rgbSource, int rgb, float[] hsbPixel)
     
    static final int[]
    argbComponents(int argb)
    Breaks a Processing color into A, R, G and B values in an array.
    static float
    audioToHSBFloat(float val)
    Converts a float value in the range [-1, 1] to a float value in the range [0, 1.0].
    static int
    audioToRGBChan(float val)
    Converts a float value in the range [-1.0, 1.0] to an int value in the range [0, 255].
    static float
    brightness(int rgb)
    Extracts the brightness component from an RGB value.
    static float
    brightness(int rgb, float[] hsbPixel)
    Extracts the brightness component from an RGB value.
    static int
    colorShift(int rgbColor, float shift)
     
    static String
    colorString(int argb)
     
    static final int
    composeColor(int[] comp)
    Creates a Processing ARGB color from r, g, b, values in an array.
    static final int
    composeColor(int r, int g, int b)
    Creates an opaque Processing RGB color from r, g, b values.
    static final int
    composeColor(int r, int g, int b, int a)
    Creates a Processing ARGB color from r, g, b, and alpha channel values.
    float[]
    conformArray(float[] source)
     
    static float[]
    conformArray(float[] source, int mapSize)
     
    int[]
    conformArray(int[] source)
    For use with the audio to color transcoding in PixelAudioMapper, Int[] and float[] arrays must be formatted as 24- or 32-bit RGB data.
    static int[]
    conformArray(int[] source, int mapSize)
     
    static float
    extractColorAsAudio(int rgb, PixelAudioMapper.ChannelNames chan, float[] hsbPixel)
    Helper method for color to audio operations, converts a color channel value to an audio value.
    PixelMapGen
    getGenerator()
     
    ArrayList<int[]>
    getGeneratorCoordinatesCopy()
     
    String
    getGeneratorDescription()
     
    int
    getHeight()
     
    int[]
    getImageToSignalLUT()
     
    int[]
    getInverseArray(int[] src)
    Call only on arrays containing the values 0..array.length-1, which can be used to reorder bitmaps.
    static final int
    getLuminosity(int rgb)
     
    int[]
    getSignalToImageLUT()
     
    int
    getSize()
     
    int
    getWidth()
     
    static float
    hsbFloatToAudio(float val)
    Converts a float value in the range [0, 1] to a float value in the range [-1.0, 1.0].
    static float
    hue(int rgb)
    Extracts the hue component from an RGB value.
    static float
    hue(int rgb, float[] hsbPixel)
    Extracts the hue component from an RGB value.
    int[]
    lookupCoordinate(int signalPos)
    Given an index into a signal array mapped to an image, returns the pixel coordinates (x,y) in the image using the lookup table signalToImageLUT.
    int
    lookupPixel(int signalPos)
    Given an index into a signal array mapped to an image, returns the corresponding index into the image pixel array using the lookup table signalToImageLUT.
    int
    lookupSample(int imagePos)
    Given an index imagePos into the pixel array of an image, returns its index in a signal path over the image using the lookup table imageToSignalLUT.
    int
    lookupSample(int x, int y)
    Given a coordinate pair (x,y) in an image, returns its index in a signal path over the image using the lookup table imageToSignalLUT.
    float[]
    mapImgToSig(int[] img, float[] sig)
    Map current image pixel values to the signal, updating the signal array.
    float[]
    mapImgToSig(int[] img, float[] sig, PixelAudioMapper.ChannelNames fromChannel)
    Map current image pixel values to the signal, updating the signal array, deriving a value from specified color channel of the image.
    int[]
    mapSigToImg(float[] sig, int[] img)
    Map signal values to the image using all channels (effectively, grayscale).
    int[]
    mapSigToImg(float[] sig, int[] img, PixelAudioMapper.ChannelNames toChannel)
    Map signal values to a specified channel in the image using imageToSignalLUT.
    int[]
    peelPixels(int[] img, int x, int y, int w, int h)
    Copies a rectangular area of pixels in image (row-major) order and returns it as an array of RGB values (a standard operation).
    float[]
    peelPixelsAsAudio(int[] img, int x, int y, int w, int h)
    Copies a rectangular area of pixels in image (row-major) order and returns it as an array of audio values ([-1.0, 1.0]).
    float[]
    peelSamples(float[] sig, int x, int y, int w, int h)
    Copies a rectangular area of audio values from a signal mapped to an image using imageToSignalLUT to index values.
    int[]
    peelSamplesAsRGB(float[] sig, int x, int y, int w, int h)
    Copies a rectangular area of audio values from a signal mapped to an image using imageToSignalLUT to index values.
    void
    plantPixels(float[] sprout, int[] img, int signalPos, int length, PixelAudioMapper.ChannelNames toChannel)
    Writes values from audio data array sprout into the specified channel of the img array at positions mapped by the signal path, starting at signalPos for the given length.
    void
    plantPixels(int[] sprout, int[] img, int signalPos, int length)
    Starting at signalPos, writes length values from RGB array sprout into RGB array img, in signal order.
    void
    plantPixels(int[] sprout, int[] img, int signalPos, int length, PixelAudioMapper.ChannelNames toChannel)
    Writes values from RGB source array sprout into img at positions mapped by the signal path, using the specified channel.
    void
    plantSamples(float[] sprout, float[] sig, int signalPos, int length)
    Starting at signalPos, insert length audio samples from source array sprout into target array of audio samples sig.
    void
    plantSamples(int[] sprout, float[] sig, int signalPos, int length)
    Starting at signalPos, insert length transcoded RGB samples from source array sprout into target array of audio samples sig.
    void
    plantSamples(int[] sprout, float[] sig, int signalPos, int length, PixelAudioMapper.ChannelNames fromChannel)
    Writes transcoded values from a specified channel of a color array (sprout) into an audio array (sig) starting at signalPos for the given length.
    int[]
    pluckPixels(int[] img, int signalPos, int length)
    Starting at signalPos, reads length values from pixel array img in signal path order using signalToImageLUT to redirect indexing and then returns them as an array of RGB pixel values in signal order.
    float[]
    pluckPixelsAsAudio(int[] img, int signalPos, int length, PixelAudioMapper.ChannelNames fromChannel)
    Starting at signalPos, reads length values from pixel array img in signal order using signalToImageLUT to redirect indexing and then returns them as an array of transcoded float values.
    float[]
    pluckSamples(float[] sig, int signalPos, int length)
    Starting at signalPos, reads length values from float array sig and returns them as a new array of audio values in signal order.
    int[]
    pluckSamplesAsRGB(float[] sig, int signalPos, int length)
    Starting at signalPos, reads and transcodes length values from float array sig and returns them as an RGB array in signal order.
    static int[]
    pullAudioAsColor(float[] sig)
     
    static int[]
    pullAudioAsColor(float[] samples, int[] rgbPixels, int signalPos, int length)
     
    static float[]
    pullPixelAsAudio(int[] rgbPixels, float[] samples, int[] lut, PixelAudioMapper.ChannelNames chan, float[] hsbPixel)
    Converts an array of pixel channel values to an array of audio sample values, mapping sample values to the interval [-1.0, 1.0], using a supplied lookup table to change the order of resulting array.
    static float[]
    pullPixelAsAudio(int[] rgbPixels, float[] samples, PixelAudioMapper.ChannelNames chan, float[] hsbPixel)
    Converts an array of pixel channel values to an array of audio sample values, mapping sample values to the interval [-1.0, 1.0], with no remapping of array order.
    static int[]
    pushAudioToChannel(float[] samples, int[] rgbPixels, int[] lut, PixelAudioMapper.ChannelNames chan)
    Replaces a specified channel in an array of pixel values, rgbPixels, with a value derived from an array of floats, buf, that represent audio samples.
    static int[]
    pushAudioToPixel(float[] samples, int[] rgbPixels, PixelAudioMapper.ChannelNames chan)
    Replaces a specified channel in an array of pixel values, rgbPixels, with a value derived from an array of floats, buf, that represent audio samples.
    static int[]
    pushChannelToPixel(int[] colors, int[] rgbPixels, int[] lut, PixelAudioMapper.ChannelNames chan)
    Replaces a specified channel in an array of pixel values, rgbPixels, with a value derived from an array of RGB values, colors.
    static int[]
    pushChannelToPixel(int[] colors, int[] rgbPixels, PixelAudioMapper.ChannelNames chan)
    Replaces a specified channel in an array of pixel values, rgbPixels, with a value derived from another array of RGB values, colors.
    void
    regenerate()
    Calls PixelMapGen generator to create coordinates and LUTs.
    static int[]
    remapPixels(int[] img, int[] lut)
    Creates an array of int which contains the values in img reordered by the lookup table lut.
    static float[]
    remapSamples(float[] sig, int[] lut)
    Creates an array of float which contains the values in sig reordered by the lookup table lut.
    static final void
    reverseArray(float[] arr, int l, int r)
    Reverses an arbitrary subset of an array of floats.
    static final void
    reverseArray(int[] arr, int l, int r)
    Reverses an arbitrary subset of an array of ints.
    static final int[]
    rgbaComponents(int argb)
    Breaks a Processing color into R, G, B and A values in an array.
    static float
    rgbChanToAudio(int val)
    Converts an int value in the range [0..255] to a float value in the range [-1.0, 1.0].
    static final int[]
    rgbComponents(int rgb)
    Breaks a Processing color into R, G and B values in an array.
    static float
    rgbFloatToAudio(float val)
    Converts a float value in the range [0, 255] to a float value in the range [-1.0, 1.0].
    static final void
    rotateLeft(float[] arr, int d)
    Rotates an array of floats left by d values.
    static final void
    rotateLeft(int[] arr, int d)
    Rotates an array of ints left by d values.
    static final void
    rotateRight(float[] arr, int d)
     
    static final void
    rotateRight(int[] arr, int d)
     
    static float
    saturation(int rgb)
    Extracts the saturation component from an RGB value.
    static float
    saturation(int rgb, float[] hsbPixel)
    Extracts the saturation component from an RGB value.
    static final int[]
    setAlpha(int[] argb, int alpha)
     
    static final int
    setAlpha(int argb, int alpha)
     
    void
    setGenerator(PixelMapGen newGen)
    Sets the PixelMapGen generator for this PixelAudioMapper.
    protected void
    setImageToSignalLUT(int[] imgLUT)
    Sets a new lookup table for mapping image to signal.
    protected void
    setSignalToImageLUT(int[] sigLUT)
    Sets a new lookup table for mapping signal to image.
    void
    stampPixels(float[] stamp, int[] img, int x, int y, int w, int h, PixelAudioMapper.ChannelNames toChannel)
    Pastes a source array of audio data into a specified color channel of a rectangular area of a destination image.
    void
    stampPixels(int[] stamp, int[] img, int x, int y, int w, int h)
    Pastes a source array of RGB data into a rectangular area of a destination image (a standard operation).
    void
    stampPixels(int[] stamp, int[] img, int x, int y, int w, int h, PixelAudioMapper.ChannelNames toChannel)
    Pastes a specified channel of a source array of RGB data into a rectangular area of a destination image (a standard operation).
    void
    stampSamples(float[] stamp, float[] sig, int x, int y, int w, int h)
    Pastes a source array of audio data into a destination array of audio data using imagetoSignalLUT to map data from source to destination.
    void
    stampSamples(int[] stamp, float[] sig, int x, int y, int w, int h)
    Pastes a source array of audio data into a destination array of RGB data as grayscale luminosity values.
    String
    toString()
     
    void
    writeImgToSig(int[] img, float[] sig)
    Writes transcoded pixel values directly to the signal, without using a LUT to redirect.
    void
    writeImgToSig(int[] img, float[] sig, PixelAudioMapper.ChannelNames fromChannel)
     
    void
    writeSigToImg(float[] sig, int[] img)
     
    void
    writeSigToImg(float[] sig, int[] img, PixelAudioMapper.ChannelNames toChannel)
     

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

  • Field Details

    • width

      protected int width
      image width

    • height

      protected int height
      image height

    • mapSize

      protected int mapSize
      pixel array and signal array length, equal to w * h

    • signalToImageLUT

      protected int[] signalToImageLUT
      Lookup table to go from the signal to the image: index values over {0..(h * w - 1)} point to a corresponding index position in the image array img.pixels[]

    • imageToSignalLUT

      protected int[] imageToSignalLUT
      Lookup table to go from the image to the signal: index values over {0..(h * w - 1)} point to a corresponding index position in the signal array sig[]

    • generator

      protected PixelMapGen generator
      PixelMapGenINF instance to generate LUTs

  • Constructor Details

    • PixelAudioMapper

      public PixelAudioMapper(PixelMapGen gen)
      Basic constructor for PixelAudio, sets up all variables.
      Parameters:
      gen - A PixelMapGenINF instance -- should be initialized already.

  • Method Details

    • getWidth

      public int getWidth()
      Returns:
      the width of the image

    • getHeight

      public int getHeight()
      Returns:
      the height of the image

    • getSize

      public int getSize()
      Returns:
      the length of the signal array (== length of image pixel array and the LUTs)

    • toString

      public String toString()
      Overrides:
      toString in class Object
      Returns:
      a string representation of our data, possibly partial

    • getSignalToImageLUT

      public int[] getSignalToImageLUT()
      Returns:
      the lookup table that maps an index in the signal to the corresponding pixel index in the image.

    • setSignalToImageLUT

      protected void setSignalToImageLUT(int[] sigLUT)
      Sets a new lookup table for mapping signal to image. Warning: The size of sigLUT must conform to the size the current image and signal arrays.
      Parameters:
      sigLUT -

    • getImageToSignalLUT

      public int[] getImageToSignalLUT()
      Returns:
      the lookup table that maps pixel values in the image to the corresponding entry in the signal.

    • setImageToSignalLUT

      protected void setImageToSignalLUT(int[] imgLUT)
      Sets a new lookup table for mapping image to signal. Warning: the size of imgLUT must conform to the size the current image and signal arrays.
      Parameters:
      imgLUT -

    • getGeneratorCoordinatesCopy

      public ArrayList<int[]> getGeneratorCoordinatesCopy()
      Returns:
      a copy of the coordinates of the signal path from the PixelMapGen generator.

    • getGeneratorDescription

      public String getGeneratorDescription()
      Returns:
      the descriptive string associated with the PixelMapGen generator

    • getGenerator

      public PixelMapGen getGenerator()
      Returns:
      the PixelMapGen generator for this PixelAudioMapper

    • setGenerator

      public void setGenerator(PixelMapGen newGen)
      Sets the PixelMapGen generator for this PixelAudioMapper.
      Parameters:
      newGen - a new PixelMapGen

    • regenerate

      public void regenerate()
      Calls PixelMapGen generator to create coordinates and LUTs.

    • getInverseArray

      public int[] getInverseArray(int[] src)
      Call only on arrays containing the values 0..array.length-1, which can be used to reorder bitmaps. The array returned will restore the order.

    • audioToRGBChan

      public static int audioToRGBChan(float val)
      Converts a float value in the range [-1.0, 1.0] to an int value in the range [0, 255]. Using 127.5f and 0.5f as values works around a call to Math.round(), saving a few cycles.
      Parameters:
      val - a float value in the range [-1.0, 1.0]
      Returns:
      an int mapped to the range [0, 255]

    • rgbChanToAudio

      public static float rgbChanToAudio(int val)
      Converts an int value in the range [0..255] to a float value in the range [-1.0, 1.0].
      Parameters:
      val - an int in the range [0..255]
      Returns:
      a float mapped to the range [-1.0, 1.0]

    • rgbFloatToAudio

      public static float rgbFloatToAudio(float val)
      Converts a float value in the range [0, 255] to a float value in the range [-1.0, 1.0].
      Parameters:
      val - a float in the range [0.0, 255.0], RGB channel value as a float
      Returns:
      a float mapped to the range [-1.0, 1.0]

    • hsbFloatToAudio

      public static float hsbFloatToAudio(float val)
      Converts a float value in the range [0, 1] to a float value in the range [-1.0, 1.0].
      Parameters:
      val - a float in the range [0..1] that represents an HSB component
      Returns:
      a float mapped to the range [-1.0, 1.0]

    • audioToHSBFloat

      public static float audioToHSBFloat(float val)
      Converts a float value in the range [-1, 1] to a float value in the range [0, 1.0].
      Parameters:
      val - a float in the range [-1..1] that typically represents an audio sample
      Returns:
      a float mapped to the range [0, 1.0], typically used in the HSB color space

    • lookupSample

      public int lookupSample(int x, int y)
      Given a coordinate pair (x,y) in an image, returns its index in a signal path over the image using the lookup table imageToSignalLUT.
      Parameters:
      x -
      y -
      Returns:
      index into signal array

    • lookupSample

      public int lookupSample(int imagePos)
      Given an index imagePos into the pixel array of an image, returns its index in a signal path over the image using the lookup table imageToSignalLUT.
      Parameters:
      imagePos -
      Returns:
      index into signal array

    • lookupPixel

      public int lookupPixel(int signalPos)
      Given an index into a signal array mapped to an image, returns the corresponding index into the image pixel array using the lookup table signalToImageLUT.
      Parameters:
      signalPos -
      Returns:

    • lookupCoordinate

      public int[] lookupCoordinate(int signalPos)
      Given an index into a signal array mapped to an image, returns the pixel coordinates (x,y) in the image using the lookup table signalToImageLUT.
      Parameters:
      signalPos -
      Returns:
      an array of two coordinates {x, y}

    • remapPixels

      public static int[] remapPixels(int[] img, int[] lut)
      Creates an array of int which contains the values in img reordered by the lookup table lut. The two arrays, img and lut, must be the same size.
      Parameters:
      img - an array of int, typically of RGB values
      lut - a look up table of the same size as img
      Returns:
      a new array of int with the values in img reordered by the lookup table

    • remapSamples

      public static float[] remapSamples(float[] sig, int[] lut)
      Creates an array of float which contains the values in sig reordered by the lookup table lut. The two arrays, sig and lut, must be the same size.
      Parameters:
      sig - an array of float, typically audio samples
      lut - a lookup table
      Returns:
      a new array with the values in sig reordered by the lookup table

    • mapSigToImg

      public int[] mapSigToImg(float[] sig, int[] img)
      Map signal values to the image using all channels (effectively, grayscale). On completion, img[] contains new values. The img array and the sig array must be the same size.
      Parameters:
      sig - source array of floats in the audio range [-1.0, 1.0]
      img - target array of RGB pixel values
      Returns:
      array of RGB int values derived from sig, loaded to all channels (audio as grayscale)

    • mapSigToImg

      public int[] mapSigToImg(float[] sig, int[] img, PixelAudioMapper.ChannelNames toChannel)
      Map signal values to a specified channel in the image using imageToSignalLUT. On completion, img[] contains new values, transcoded from the signal. The img array and the sig array must be the same size.
      Parameters:
      sig - an array of floats in the audio range [-1.0, 1.0]
      img - an array of RGB pixel values
      toChannel - the channel to write transcoded values to
      Returns:
      array of RGB int values derived from sig, loaded to specified channel

    • mapImgToSig

      public float[] mapImgToSig(int[] img, float[] sig)
      Map current image pixel values to the signal, updating the signal array. There are several ways to derive an audio value from the image: we use the brightness channel in the HSB color space. On completion, sig[] contains new values. The img array and the sig array must be the same size.
      Parameters:
      sig - an array of floats in the audio range [-1.0, 1.0]
      img - an array of RGB pixel values
      Returns:
      array of audio range float values derived from Brightness channel of color values

    • mapImgToSig

      public float[] mapImgToSig(int[] img, float[] sig, PixelAudioMapper.ChannelNames fromChannel)
      Map current image pixel values to the signal, updating the signal array, deriving a value from specified color channel of the image. On completion, sig[] contains new values. The img array and the sig array must be the same size.
      Parameters:
      sig - an array of floats in the audio range [-1.0, 1.0]
      img - an array of RGB pixel values
      fromChannel - the color channel to get a value from
      hsbPixel - a float[3] array for use with color channel extraction
      Returns:
      array of audio range float values derived from specified channel of color values

    • writeImgToSig

      public void writeImgToSig(int[] img, float[] sig)
      Writes transcoded pixel values directly to the signal, without using a LUT to redirect. Values are calculated with the standard luminosity equation, gray = 0.3 * red + 0.59 * green + 0.11 * blue.
      Parameters:
      img - source array of RGB pixel values
      sig - target array of audio samples in the range [-1.0, 1.0]

    • writeImgToSig

      public void writeImgToSig(int[] img, float[] sig, PixelAudioMapper.ChannelNames fromChannel)
      Parameters:
      img - an array of RGB pixel values, source
      sig - an array of audio samples in the range [-1.0, 1.0], target
      fromChannel - channel in RGB or HSB color space, from ChannelNames enum

    • writeSigToImg

      public void writeSigToImg(float[] sig, int[] img)
      Parameters:
      sig - an array of audio samples in the range [-1.0, 1.0], source
      img - an array of RGB pixel values, target

    • writeSigToImg

      public void writeSigToImg(float[] sig, int[] img, PixelAudioMapper.ChannelNames toChannel)
      Parameters:
      sig - an array of audio samples in the range [-1.0, 1.0], source
      img - an array of RGB pixel values, target
      toChannel - channel in RGB or HSB color space, from ChannelNames enum

    • conformArray

      public int[] conformArray(int[] source)
      For use with the audio to color transcoding in PixelAudioMapper, Int[] and float[] arrays must be formatted as 24- or 32-bit RGB data. Float[] arrays must be formatted as audio [-1.0, 1.0] data. The arrays must conform to the dimensions of the PixelAudioMapper instance, so that array.length == this.mapSize. We do NOT check for null arrays.

    • conformArray

      public float[] conformArray(float[] source)

    • conformArray

      public static int[] conformArray(int[] source, int mapSize)

    • conformArray

      public static float[] conformArray(float[] source, int mapSize)

    • pluckPixels

      public int[] pluckPixels(int[] img, int signalPos, int length)
      Starting at signalPos, reads length values from pixel array img in signal path order using signalToImageLUT to redirect indexing and then returns them as an array of RGB pixel values in signal order. Note that signalPos = this.imageToSignalLUT[x + y * this.width]. The source image data in img must conform to the current PixelAudioMapper instance's dimensions, otherwise we'll throw an IllegalArgumentException.
      Parameters:
      img - array of RGB values, the pixels array from a bitmap image with the same width and height as PixelAudioMapper
      signalPos - position in the signal at which to start reading pixel values from the image, following the signal path
      length - length of the subarray to pluck from img, reading pixel values while following the signal path
      Returns:
      a new array of pixel values in signal order
      Throws:
      IllegalArgumentException - if parameters are out of bounds or arrays are null, or if img.length != this.width * this.height

    • pluckPixelsAsAudio

      public float[] pluckPixelsAsAudio(int[] img, int signalPos, int length, PixelAudioMapper.ChannelNames fromChannel)
      Starting at signalPos, reads length values from pixel array img in signal order using signalToImageLUT to redirect indexing and then returns them as an array of transcoded float values. Note that signalPos = this.imageToSignalLUT[x + y * this.width] or this.lookupSample(x, y). Starting at image coordinates (x, y), reads values from pixel array img using imageToSignalLUT to redirect indexing and returns them as an array of transcoded audio values in signal order.
      Parameters:
      img - source array of RGB pixel values, must conform to PixelAudioMapper dimensions
      signalPos - position in the signal at which to start reading pixel values from the image, following the signal path
      length - length of the subarray to pluck from img
      fromChannel - the color channel from which to read pixel values
      Returns:
      a new array of audio values in signal order
      Throws:
      IllegalArgumentException - if parameters are out of bounds or arrays are null, or if img.length != this.width * this.height

    • pluckSamples

      public float[] pluckSamples(float[] sig, int signalPos, int length)
      Starting at signalPos, reads length values from float array sig and returns them as a new array of audio values in signal order. Really just a standard subarray copy method. No lookup tables are used. Does not require array lengths to equal this.width * this.height. All we're doing is getting a subarray of an array of float. Since we don't use indirect indexing with LUTs, sig.length is not required to equal this.mapSize.
      Parameters:
      sig - source array of audio values
      signalPos - a position in the sig array
      length - number of values to read from sig array
      Returns:
      a new array with the audio values we read
      Throws:
      IllegalArgumentException - if parameters are out of bounds or arrays are null

    • pluckSamplesAsRGB

      public int[] pluckSamplesAsRGB(float[] sig, int signalPos, int length)
      Starting at signalPos, reads and transcodes length values from float array sig and returns them as an RGB array in signal order. No lookup tables are used. Does not require array lengths to equal this.width * this.height. We're getting a subarray of an array of float and transcoding it to an array of RGB int values. Since we don't use indirect indexing with LUTs, sig.length is not required to equal this.mapSize.
      Parameters:
      sig - source array of audio values (-1.0f..1.0f)
      signalPos - entry point in the sig array
      length - number of values to read from the sig array
      Returns:
      an array of RGB values where r == g == b, derived from the sig values
      Throws:
      IllegalArgumentException - if parameters are out of bounds or arrays are null

    • plantPixels

      public void plantPixels(int[] sprout, int[] img, int signalPos, int length)
      Starting at signalPos, writes length values from RGB array sprout into RGB array img, in signal order. Since we redirect indexing with a lookup table, img.length is necessarily equal to mapSize, i.e., this.width * this.height. If not, we'll throw an IllegalArgumentException.
      Parameters:
      sprout - source array of RGB values to insert into target array img
      img - target array of RGB values. in image (row major) order
      signalPos - position in the signal at which to start writing pixel values to the image, following the signal path
      length - number of values from sprout to insert into img array
      Throws:
      IllegalArgumentException - if parameters are out of bounds or arrays are null, or if img.length != this.width * this.height

    • plantPixels

      public void plantPixels(int[] sprout, int[] img, int signalPos, int length, PixelAudioMapper.ChannelNames toChannel)
      Writes values from RGB source array sprout into img at positions mapped by the signal path, using the specified channel. Since we redirect indexing with a lookup table, img.length is necessarily equal to mapSize, i.e., this.width * this.height. If not, we'll throw an IllegalArgumentException.
      Parameters:
      sprout - source array of RGB values to insert
      img - target array of RGB values (image, row-major order)
      signalPos - signal position to start writing
      length - number of values to write
      toChannel - channel to write into (R, G, B, L, etc.)
      Throws:
      IllegalArgumentException - if parameters are out of bounds or arrays are null, or if img.length != this.width * this.height

    • plantPixels

      public void plantPixels(float[] sprout, int[] img, int signalPos, int length, PixelAudioMapper.ChannelNames toChannel)
      Writes values from audio data array sprout into the specified channel of the img array at positions mapped by the signal path, starting at signalPos for the given length. Since we redirect indexing with a lookup table, img.length is necessarily equal to mapSize, i.e., this.width * this.height. If not, we'll throw an IllegalArgumentException.
      Parameters:
      sprout - source array audio samples ([-1.0, 1.0])
      img - target array of RGB values (image, row-major order)
      signalPos - signal position to start writing
      length - number of values to write
      toChannel - color channel to write to
      Throws:
      IllegalArgumentException - if parameters are out of bounds or arrays are null, or if img.length != this.width * this.height

    • plantSamples

      public void plantSamples(float[] sprout, float[] sig, int signalPos, int length)
      Starting at signalPos, insert length audio samples from source array sprout into target array of audio samples sig. In effect, a subarray insertion method. No lookup tables are used. Does not require array lengths to equal this.width * this.height.
      Parameters:
      sprout - source array of audio values (-1.0f..1.0f)
      sig - target array of signal values, in signal order
      signalPos - start point in sig array
      length - number of values to copy from sprout array to sig array
      Throws:
      IllegalArgumentException - if parameters are out of bounds or arrays are null

    • plantSamples

      public void plantSamples(int[] sprout, float[] sig, int signalPos, int length)
      Starting at signalPos, insert length transcoded RGB samples from source array sprout into target array of audio samples sig. No lookup tables are used. Does not require array lengths to equal this.width * this.height.
      Parameters:
      sprout - source array of RGB color values
      sig - target array of audio values
      signalPos - insertion point in sig
      length - number of values to write to sig
      Throws:
      IllegalArgumentException - if parameters are out of bounds or arrays are null

    • plantSamples

      public void plantSamples(int[] sprout, float[] sig, int signalPos, int length, PixelAudioMapper.ChannelNames fromChannel)
      Writes transcoded values from a specified channel of a color array (sprout) into an audio array (sig) starting at signalPos for the given length. No lookup tables are used. Does not require array lengths to equal this.width * this.height.
      Parameters:
      sprout - source array of RGB pixel values
      sig - target array of audio samples (float, [-1.0, 1.0])
      signalPos - position to start writing into sig
      length - number of values to write
      fromChannel - channel to extract from sprout values
      Throws:
      IllegalArgumentException - if parameters are out of bounds or arrays are null

    • peelPixels

      public int[] peelPixels(int[] img, int x, int y, int w, int h)
      Copies a rectangular area of pixels in image (row-major) order and returns it as an array of RGB values (a standard operation). No lookup tables are used. Requires img.length to equal this.width * this.height.
      Parameters:
      img - the image pixel array (row-major, length == width * height)
      x - left edge of rectangle
      y - top edge of rectangle
      w - width of rectangle
      h - height of rectangle
      Returns:
      array of int (RGB values), length w*h
      Throws:
      IllegalArgumentException - if parameters are out of bounds or arrays are null or if img.length != this.width * this.height

    • peelPixelsAsAudio

      public float[] peelPixelsAsAudio(int[] img, int x, int y, int w, int h)
      Copies a rectangular area of pixels in image (row-major) order and returns it as an array of audio values ([-1.0, 1.0]). No lookup tables are used. Requires img.length to equal this.width * this.height.
      Parameters:
      img - the image pixel array (row-major, length == width * height)
      x - left edge of rectangle
      y - top edge of rectangle
      w - width of rectangle
      h - height of rectangle
      Returns:
      array of float (audio values), length w*h
      Throws:
      IllegalArgumentException - if parameters are out of bounds or arrays are null or if img.length != this.width * this.height

    • peelSamples

      public float[] peelSamples(float[] sig, int x, int y, int w, int h)
      Copies a rectangular area of audio values from a signal mapped to an image using imageToSignalLUT to index values. With the resulting array you could, for example, run a 2D filter over selected 1D audio data. Requires sig.length to equal this.width * this.height.
      Parameters:
      sig - a source array of audio samples ([-1.0, 1.0])
      x - left edge of rectangle
      y - top edge of rectangle
      w - width of rectangle
      h - height of rectangle
      Returns:
      array of float (audio values), length w*h
      Throws:
      IllegalArgumentException - if parameters are out of bounds or arrays are null, or if sig.length != this.width * this.height

    • peelSamplesAsRGB

      public int[] peelSamplesAsRGB(float[] sig, int x, int y, int w, int h)
      Copies a rectangular area of audio values from a signal mapped to an image using imageToSignalLUT to index values. Requires sig.length to equal this.width * this.height.
      Parameters:
      sig - a source array of audio values ([-1.0, 1.0])
      x - left edge of rectangle
      y - top edge of rectangle
      w - width of rectangle
      h - height of rectangle
      Returns:
      array of float (audio values), length w*h
      Throws:
      IllegalArgumentException - if parameters are out of bounds or arrays are null, or if sig.length != this.width * this.height

    • stampPixels

      public void stampPixels(int[] stamp, int[] img, int x, int y, int w, int h)
      Pastes a source array of RGB data into a rectangular area of a destination image (a standard operation). No lookup tables are used. Requires img.length to equal this.width * this.height.
      Parameters:
      stamp - a source array of RGB data
      img - a destination image
      x - leftmost x-coordinate of a rectangular area in the destination image
      y - topmost y-coordinate of a rectangular area in the destination image
      w - width of rectangular area
      h - height of rectangular area
      Throws:
      IllegalArgumentException - if parameters are out of bounds or arrays are null or if img.length != this.width * this.height

    • stampPixels

      public void stampPixels(int[] stamp, int[] img, int x, int y, int w, int h, PixelAudioMapper.ChannelNames toChannel)
      Pastes a specified channel of a source array of RGB data into a rectangular area of a destination image (a standard operation). No lookup tables are used. Requires img.length to equal this.width * this.height.
      Parameters:
      stamp - a source array of RGB data
      img - a destination image
      x - leftmost x-coordinate of a rectangular area in the destination image
      y - topmost y-coordinate of a rectangular area in the destination image
      w - width of rectangular area
      h - height of rectangular area
      toChannel - color channel to write to
      Throws:
      IllegalArgumentException - if parameters are out of bounds or arrays are null or if img.length != this.width * this.height

    • stampPixels

      public void stampPixels(float[] stamp, int[] img, int x, int y, int w, int h, PixelAudioMapper.ChannelNames toChannel)
      Pastes a source array of audio data into a specified color channel of a rectangular area of a destination image. No lookup tables are used. Requires img.length to equal this.width * this.height.
      Parameters:
      stamp - a source array of audio data ([-1.0, 1.0])
      img - a destination image
      x - leftmost x-coordinate of a rectangular area in the destination image
      y - topmost y-coordinate of a rectangular area in the destination image
      w - width of rectangular area
      h - height of rectangular area
      toChannel - color channel to write to
      Throws:
      IllegalArgumentException - if parameters are out of bounds or arrays are null or if img.length != this.width * this.height

    • stampSamples

      public void stampSamples(float[] stamp, float[] sig, int x, int y, int w, int h)
      Pastes a source array of audio data into a destination array of audio data using imagetoSignalLUT to map data from source to destination. In effect, source and destination are treated as 2D rectangular arrays. Note that sig.length must equal this.width * this.height.
      Parameters:
      stamp - a source array of audio data ([-1.0, 1.0])
      sig - a destination array of audio data
      x - leftmost x-coordinate of a rectangular area in the destination image
      y - topmost y-coordinate of a rectangular area in the destination image
      w - width of rectangular area
      h - height of rectangular area
      Throws:
      IllegalArgumentException - if parameters are out of bounds or arrays are null, or if sig.length != this.width * this.height

    • stampSamples

      public void stampSamples(int[] stamp, float[] sig, int x, int y, int w, int h)
      Pastes a source array of audio data into a destination array of RGB data as grayscale luminosity values. Note that sig.length must equal this.width * this.height.
      Parameters:
      stamp -
      sig -
      x -
      y -
      w -
      h -
      Throws:
      IllegalArgumentException - if parameters are out of bounds or arrays are null, or if sig.length != this.width * this.height

    • extractColorAsAudio

      public static float extractColorAsAudio(int rgb, PixelAudioMapper.ChannelNames chan, float[] hsbPixel)
      Helper method for color to audio operations, converts a color channel value to an audio value.
      Parameters:
      rgb - an RGB color value
      chan - the channel to extract for the RGB color value
      hsbPixel - a local array for HSB values, will contain HSB data on completion of HSB extraction
      Returns:
      the extracted channel as an audio float value

    • applyAudioToColor

      public static int applyAudioToColor(float sample, int rgb, PixelAudioMapper.ChannelNames chan, float[] hsbPixel)
      Helper method that applies a float audio sample to a pixel channel, using the channel name.
      Parameters:
      sample - audio float source of color value
      rgb - destination RGB color value
      chan - the color channel to use
      hsbPixel - array of 3 floats to maintain HSB color data
      Returns:
      RGB color derived from rgb, with color channel changed

    • applyChannelToColor

      public static int applyChannelToColor(int rgbSource, int rgbTarget, PixelAudioMapper.ChannelNames chan, float[] hsbPixel)
      Helper method for applying a color channel from a to an RGB pixel.
      Parameters:
      sample - RGB source of color value
      rgb - destination color value
      chan - the color channel to use
      hsbPixel - array of 3 floats to maintain HSB color data
      Returns:

    • pullPixelAsAudio

      public static float[] pullPixelAsAudio(int[] rgbPixels, float[] samples, PixelAudioMapper.ChannelNames chan, float[] hsbPixel)
      Converts an array of pixel channel values to an array of audio sample values, mapping sample values to the interval [-1.0, 1.0], with no remapping of array order. If samples is null or samples.length != rgbPixels.length, initializes/adjusts samples.length.
      Parameters:
      rgbPixels - an array of RGB pixel values
      samples - an array of audio samples, which may be null, whose values will be set from rgbPixels
      chan - channel to extract from the RGB pixel values Will be initialized and returned if null
      Returns:
      a array of floats mapped to the audio range, assigned to samples

    • pullPixelAsAudio

      public static float[] pullPixelAsAudio(int[] rgbPixels, float[] samples, int[] lut, PixelAudioMapper.ChannelNames chan, float[] hsbPixel)
      Converts an array of pixel channel values to an array of audio sample values, mapping sample values to the interval [-1.0, 1.0], using a supplied lookup table to change the order of resulting array. If samples is null or samples.length != rgbPixels.length, initializes/adjusts samples.length.
      Parameters:
      rgbPixels - an array of RGB pixel values
      samples - an array of audio samples, which may be null, whose values will be set from rgbPixels.
      lut - a lookup table for redirecting rgbPixels indexing, typically imageToSignalLUT
      chan - channel to extract from the RGB pixel values Will be initialized and returned if null.
      Returns:
      a array of floats mapped to the audio range, identical to samples

    • pullAudioAsColor

      public static int[] pullAudioAsColor(float[] samples, int[] rgbPixels, int signalPos, int length)

    • pullAudioAsColor

      public static int[] pullAudioAsColor(float[] sig)

    • pushAudioToPixel

      public static int[] pushAudioToPixel(float[] samples, int[] rgbPixels, PixelAudioMapper.ChannelNames chan)

      Replaces a specified channel in an array of pixel values, rgbPixels, with a value derived from an array of floats, buf, that represent audio samples. Upon completion, the pixel array rgbPixels contains the new values, always in the RGB color space.

      Both arrays, rgbPixels and buf, must be the same size.

      In the HSB color space, values are assumed to be floats in the range (0..1), so the values from buf need to be mapped to the correct ranges for HSB or RGB [0, 255]. We do some minimal limiting of values derived from buf[], but it is the caller's responsibility to constrain them to the audio range [-1.0, 1.0].

      Parameters:
      rgbPixels - a source array of pixel values
      samples - a target array of floats in the range (-1.0, 1.0)
      chan - the channel to replace
      Returns:
      rgbPixels with the selected channel modified by the buf values

    • pushAudioToChannel

      public static int[] pushAudioToChannel(float[] samples, int[] rgbPixels, int[] lut, PixelAudioMapper.ChannelNames chan)

      Replaces a specified channel in an array of pixel values, rgbPixels, with a value derived from an array of floats, buf, that represent audio samples. The supplied lookup table, lut, is intended to redirect the indexing of rgbPixels following the signal path. We are stepping through the buf array (the signal), so rgbPixels employs imageToSignalLUT to find where each index i into buf is pointing in the image pixels array, which is rgbPixels. Upon completion, the pixel array rgbPixels contains the new values, always in the RGB color space.

      All three arrays, rgbPixels, buf, and lut must be the same size.

      In the HSB color space, values are assumed to be floats in the range (0..1), so the values from buf need to be mapped to the correct ranges for HSB or RGB [0, 255]. We do some minimal limiting of values derived from buf[], but it is the caller's responsibility to constrain them to the audio range [-1.0, 1.0].

      Parameters:
      rgbPixels - an array of pixel values
      samples - an array of floats in the range [-1.0, 1.0]
      lut - a lookup table to redirect the indexing of the buf, typically imageToPixelsLUT
      chan - the channel to replace
      Returns:
      rgbPixels with selected channel values modified by the buf values

    • pushChannelToPixel

      public static int[] pushChannelToPixel(int[] colors, int[] rgbPixels, PixelAudioMapper.ChannelNames chan)

      Replaces a specified channel in an array of pixel values, rgbPixels, with a value derived from another array of RGB values, colors. Upon completion, the pixel array rgbPixels contains the new values, always in the RGB color space.

      Both arrays, rgbPixels and colors, must be the same size.

      For operations in the HSB color space, HSB values will be extracted from the RGB colors and mapped to the correct ranges for HSB or RGB [0, 255].

      Parameters:
      colors - an array of RGB colors
      rgbPixels - an array of pixel values
      chan - the channel to replace
      Returns:
      rgbPixels with the selected channel modified by the colors array values

    • pushChannelToPixel

      public static int[] pushChannelToPixel(int[] colors, int[] rgbPixels, int[] lut, PixelAudioMapper.ChannelNames chan)

      Replaces a specified channel in an array of pixel values, rgbPixels, with a value derived from an array of RGB values, colors. The supplied lookup table, lut, is intended to redirect the indexing of rgbPixels following the signal path. We are stepping through the color array (the RGB signal), so rgbPixels employs imageToSignalLUT to find where each index i into colors is pointing in the image pixels array, rgbPixels. Upon completion, the pixel array rgbPixels contains the new values, always in the RGB color space.

      All three arrays, rgbPixels, colors, and lut must be the same size.

      In the HSB color space, HSB values will be extracted from the RGB colors and mapped to the correct ranges for HSB or RGB [0, 255].

      Parameters:
      colors - an array of RGB values
      rgbPixels - an array of pixel values
      lut - a lookup table to redirect indexing, typically imageToPixelsLUT
      chan - the channel to replace
      Returns:
      rgbPixels with selected channel values modified by the buf values

    • rotateLeft

      public static final void rotateLeft(int[] arr, int d)
      Rotates an array of ints left by d values. Uses efficient "Three Rotation" algorithm.
      Parameters:
      arr - array of ints to rotate
      d - number of elements to shift, positive for shift left, negative for shift right

    • rotateRight

      public static final void rotateRight(int[] arr, int d)

    • rotateLeft

      public static final void rotateLeft(float[] arr, int d)
      Rotates an array of floats left by d values. Uses efficient "Three Rotation" algorithm.
      Parameters:
      arr - array of floats to rotate
      d - number of elements to shift, positive for shift left, negative for shift right

    • rotateRight

      public static final void rotateRight(float[] arr, int d)

    • reverseArray

      public static final void reverseArray(int[] arr, int l, int r)
      Reverses an arbitrary subset of an array of ints.
      Parameters:
      arr - array to modify
      l - left bound of subset to reverse
      r - right bound of subset to reverse

    • reverseArray

      public static final void reverseArray(float[] arr, int l, int r)
      Reverses an arbitrary subset of an array of floats.
      Parameters:
      arr - array to modify
      l - left bound of subset to reverse
      r - right bound of subset to reverse

    • rgbComponents

      public static final int[] rgbComponents(int rgb)
      Breaks a Processing color into R, G and B values in an array.
      Parameters:
      rgb - a Processing color as a 32-bit integer
      Returns:
      an array of integers in the range [0, 255] for 3 primary color components: {R, G, B}

    • rgbaComponents

      public static final int[] rgbaComponents(int argb)
      Breaks a Processing color into R, G, B and A values in an array.
      Parameters:
      argb - a Processing color as a 32-bit integer
      Returns:
      an array of 4 integers {R, G, B, A} in the range 0..255

    • argbComponents

      public static final int[] argbComponents(int argb)
      Breaks a Processing color into A, R, G and B values in an array.
      Parameters:
      argb - a Processing color as a 32-bit integer
      Returns:
      an array of 4 integers {A, R, G, B} in the range 0..255

    • alphaComponent

      public static final int alphaComponent(int argb)
      Returns alpha channel value of a color.
      Parameters:
      argb - a Processing color as a 32-bit integer
      Returns:
      an int for alpha channel

    • applyAlpha

      public static final int applyAlpha(int rgba, int rgb)
      Takes the alpha channel from one color, rgba, and applies it to another color, rgb.
      Parameters:
      rgba - The source color from which we get the alpha channel (a mask, for instance)
      rgb - The target color we want to change
      Returns:
      A color with the RGB values from rgb and the A value from rgba

    • setAlpha

      public static final int setAlpha(int argb, int alpha)

    • setAlpha

      public static final int[] setAlpha(int[] argb, int alpha)

    • composeColor

      public static final int composeColor(int r, int g, int b, int a)
      Creates a Processing ARGB color from r, g, b, and alpha channel values. Note the order of arguments, the same as the Processing color(value1, value2, value3, alpha) method.
      Parameters:
      r - red component [0, 255]
      g - green component [0, 255]
      b - blue component [0, 255]
      a - alpha component [0, 255]
      Returns:
      a 32-bit integer with bytes in Processing format ARGB.

    • composeColor

      public static final int composeColor(int r, int g, int b)
      Creates an opaque Processing RGB color from r, g, b values. Note the order of arguments, the same as the Processing color(value1, value2, value3) method.
      Parameters:
      r - red component [0, 255]
      g - green component [0, 255]
      b - blue component [0, 255]
      Returns:
      a 32-bit integer with bytes in Processing format ARGB.

    • composeColor

      public static final int composeColor(int[] comp)
      Creates a Processing ARGB color from r, g, b, values in an array.
      Parameters:
      comp - array of 3 integers in range [0, 255], for red, green and blue components of color alpha value is assumed to be 255
      Returns:
      a 32-bit integer with bytes in Processing format ARGB.

    • getLuminosity

      public static final int getLuminosity(int rgb)
      Parameters:
      rgb - an RGB color value
      Returns:
      a number in the range [0, 255] equivalent to the luminosity value rgb

    • colorString

      public static String colorString(int argb)
      Parameters:
      argb - an RGB color
      Returns:
      a String equivalent to a Processing color(r, g, b, a) call, such as "color(233, 144, 89, 255)"

    • hue

      public static float hue(int rgb, float[] hsbPixel)
      Extracts the hue component from an RGB value. The result is in the range (0, 1).
      Parameters:
      rgb - The RGB color from which we will obtain the hue component in the HSB color model.
      hsbPixel - array of float values filled in by this method, caller can provide a reusable array
      Returns:
      A floating point number in the range (0..1) that can be multiplied by 360 to get the hue angle.

    • hue

      public static float hue(int rgb)
      Extracts the hue component from an RGB value. The result is in the range (0, 1).
      Parameters:
      rgb - The RGB color from which we will obtain the hue component in the HSB color model.
      Returns:
      A floating point number in the range (0..1) that can be multiplied by 360 to get the hue angle.

    • saturation

      public static float saturation(int rgb, float[] hsbPixel)
      Extracts the saturation component from an RGB value. The result is in the range (0, 1).
      Parameters:
      rgb - The RGB color from which we will obtain the hue component in the HSB color model.
      hsbPixel - array of float values filled in by this method
      Returns:
      A floating point number in the range (0, 1) representing the saturation component of an HSB color.

    • saturation

      public static float saturation(int rgb)
      Extracts the saturation component from an RGB value. The result is in the range (0, 1).
      Parameters:
      rgb - The RGB color from which we will obtain the hue component in the HSB color model.
      Returns:
      A floating point number in the range (0, 1) representing the saturation component of an HSB color.

    • brightness

      public static float brightness(int rgb, float[] hsbPixel)
      Extracts the brightness component from an RGB value. The result is in the range (0, 1).
      Parameters:
      rgb - The RGB color from which we will obtain the hue component in the HSB color model.
      hsbPixel - array of float values filled in by this method
      Returns:
      A floating point number in the range (0, 1) representing the brightness component of an HSB color.

    • brightness

      public static float brightness(int rgb)
      Extracts the brightness component from an RGB value. The result is in the range (0, 1).
      Parameters:
      rgb - The RGB color from which we will obtain the hue component in the HSB color model.
      Returns:
      A floating point number in the range (0, 1) representing the brightness component of an HSB color.

    • colorShift

      public static int colorShift(int rgbColor, float shift)

    • applyBrightness

      public static int applyBrightness(float sample, int rgb)

    • applyBrightness

      public static int applyBrightness(float sample, int rgb, float[] hsbPixel)

    • applyHue

      public static int applyHue(float sample, int rgb)

    • applyHue

      public static int applyHue(float sample, int rgb, float[] hsbPixel)

    • applySaturation

      public static int applySaturation(float sample, int rgb)

    • applySaturation

      public static int applySaturation(float sample, int rgb, float[] hsbPixel)

    • applyRed

      public static int applyRed(float sample, int rgb)

    • applyGreen

      public static int applyGreen(float sample, int rgb)

    • applyBlue

      public static int applyBlue(float sample, int rgb)

    • applyAlpha

      public static int applyAlpha(float sample, int rgb)

    • applyAll

      public static int applyAll(float sample, int rgb)

    • applyBrightness

      public static int applyBrightness(int rgbSource, int rgb)

    • applyBrightness

      public static int applyBrightness(int rgbSource, int rgb, float[] hsbPixel)

    • applyHue

      public static int applyHue(int rgbSource, int rgb)

    • applyHue

      public static int applyHue(int rgbSource, int rgb, float[] hsbPixel)

    • applySaturation

      public static int applySaturation(int rgbSource, int rgb)

    • applySaturation

      public static int applySaturation(int rgbSource, int rgb, float[] hsbPixel)

    • applyRed

      public static int applyRed(int rgbSource, int rgb)

    • applyGreen

      public static int applyGreen(int rgbSource, int rgb)

    • applyBlue

      public static int applyBlue(int rgbSource, int rgb)

    • applyAll

      public static int applyAll(int rgbSource, int rgb)

    • applyColor

      public static int applyColor(int colorSource, int graySource)

    • applyColor

      public static int applyColor(int colorSource, int graySource, float[] hsbPixel)

    • applyColor

      public static int[] applyColor(int[] colorSource, int[] graySource, int[] lut)
      Utility method for applying hue and saturation values from a source array of RGB values to the brightness values in a target array of RGB values, using a lookup table to redirect indexing.
      Parameters:
      colorSource - a source array of RGB data from which to obtain hue and saturation values
      graySource - an target array of RGB data from which to obtain brightness values
      lut - a lookup table, must be the same size as colorSource and graySource
      Returns:
      the graySource array of RGB values, with hue and saturation values changed
      Throws:
      IllegalArgumentException - if array arguments are null or if they are not the same length