Package net.paulhertz.pixelaudio.example

# Class LoadAudioToImage

java.lang.Object
processing.core.PApplet
net.paulhertz.pixelaudio.example.LoadAudioToImage

All Implemented Interfaces:
  `processing.core.PConstants`

public class LoadAudioToImage extends processing.core.PApplet
LoadAudioToImage shows how you can open audio files and transcode their image data to an image. Once the image is loaded, you can click on it to play back the audio using a PASamplerInstrument. Hovering over the image and pressing the spacebar will also trigger an audio event. The hightlightSample() method highlights the pixels that correspond to the audio signal that is played. The highlight changes the pixels and can change the audio, too: just press the 'w' key to transcode the image to an audio signal and write it to the PASamplerInstrument. You can also load audio to individual RGB or HSB Hue and Brightness channels. To hear the results of loading to different channels, write the image to the audio signal ('w' key) and click in the image. This sketch also includes some common methods for adjusting the brightness and contrast of an image: gamma adjustment changes contrast up of down and histogram equalization stretches the range of brightness values in an image. These operations will change the audio, too, if you write the image to the audio signal. The gamma operations both make the audio quieter, while the histogram stretch usually makes the audio louder. Press 'o' or 'O' to open an audio file and load it to the signal and all channels of the image. Press 'r' to open an audio file and load it to the signal and the RED channel of the image. Press 'g' to open an audio file and load it to the signal and the GREEN channel of the image. Press 'b' to open an audio file and load it to the signal and the BLUE channel of the image. Press 'l' to open an audio file and load it to the signal and the HSB Brightness channel of the image. Press 'h' to open an audio file and load it to the signal and the HSB Hue channel of the image. Press 'k' to overlay the rainbow signal path on the image. Press 'm' to equalize the histogram of the image. Press '+' or '=' to apply gamma value "gammaLighter" to the image. Press '-' or '_' to apply gamma value "gammaDarker" to the image. Press 'w' to transcode the image HSB Brightness channel and write it to the signal. Press 's' to save the current display image as a PNG file. Press '?' to show Help Message in the console.

- ## Field Summary

  Fields
  Modifier and Type
  Field
  Description
  `(package private) ADSRParams`
  `adsr`
  `(package private) float`
  `attackTime`
  `(package private) File`
  `audioFile`
  `(package private) String`
  `audioFileName`
  `(package private) String`
  `audioFilePath`
  `(package private) String`
  `audioFileTag`
  `(package private) int`
  `audioLength`
  `(package private) ddf.minim.AudioOutput`
  `audioOut`
  `(package private) float[]`
  `audioSignal`
  `(package private) int`
  `blau`
  `(package private) int`
  `blendAlpha`
  `(package private) PixelAudioMapper.ChannelNames`
  `chan`
  `(package private) int[]`
  `colors`
  `(package private) float`
  `decayTime`
  `(package private) int`
  `duration`
  `(package private) float`
  `gammaDarker`
  `(package private) float`
  `gammaLighter`
  `(package private) int[]`
  `gammaTable`
  `(package private) int`
  `groc`
  `(package private) HilbertGen`
  `hGen`
  `(package private) int`
  `histoHigh`
  `(package private) int`
  `histoLow`
  `(package private) processing.core.PImage`
  `mapImage`
  `(package private) PixelAudioMapper`
  `mapper`
  `(package private) int`
  `mapSize`
  `(package private) float`
  `maxAmplitude`
  `(package private) ddf.minim.Minim`
  `minim`
  Minim audio library
  `(package private) PixelAudio`
  `pixelaudio`
  PixelAudio library
  `(package private) int`
  `pixelPos`
  `(package private) ddf.minim.MultiChannelBuffer`
  `playBuffer`
  `int[]`
  `randColors`
  `(package private) Random`
  `rando`
  `(package private) float`
  `releaseTime`
  `(package private) int[]`
  `rgbSignal`
  `(package private) int`
  `roig`
  `(package private) int`
  `samplePos`
  `(package private) float`
  `sampleRate`
  `(package private) float`
  `sustainLevel`
  `(package private) PASamplerInstrument`
  `synth`
  `(package private) int`
  `taronja`
  `(package private) int`
  `vert`
  `(package private) int`
  `violet`

  ### Fields inherited from class processing.core.PApplet

  `args, ARGS_BGCOLOR, ARGS_DISABLE_AWT, ARGS_DISPLAY, ARGS_EDITOR_LOCATION, ARGS_EXTERNAL, ARGS_FULL_SCREEN, ARGS_HIDE_STOP, ARGS_LOCATION, ARGS_PRESENT, ARGS_SKETCH_FOLDER, ARGS_STOP_COLOR, ARGS_UI_SCALE, ARGS_WINDOW_COLOR, DEFAULT_HEIGHT, DEFAULT_WIDTH, disableAWT, displayHeight, displayWidth, dmouseX, dmouseY, emouseX, emouseY, exitCalled, EXTERNAL_MOVE, EXTERNAL_STOP, finished, firstMouse, focused, frameCount, frameRate, frameRateLastNanos, g, height, insideDraw, javaPlatform, javaVersion, javaVersionName, key, keyCode, keyEvent, keyPressed, keyRepeatEnabled, looping, matchPatterns, mouseButton, mouseEvent, mousePressed, mouseX, mouseY, pixelDensity, pixelHeight, pixels, pixelWidth, platform, pmouseX, pmouseY, ratioLeft, ratioScale, ratioTop, recorder, redraw, rheight, rmouseX, rmouseY, rwidth, surface, useNativeSelect, width, windowX, windowY`

  ### Fields inherited from interface processing.core.PConstants

  `ADD, ALPHA, ALT, AMBIENT, ARC, ARGB, ARROW, BACKSPACE, BASELINE, BEVEL, BEZIER_VERTEX, BLEND, BLUR, BOTTOM, BOX, BREAK, BURN, CENTER, CHATTER, CHORD, CLAMP, CLOSE, CODED, COMPLAINT, CONTROL, CORNER, CORNERS, CROSS, CURVE_VERTEX, CUSTOM, DARKEST, DEG_TO_RAD, DELETE, DIAMETER, DIFFERENCE, DILATE, DIRECTIONAL, DISABLE_ASYNC_SAVEFRAME, DISABLE_BUFFER_READING, DISABLE_DEPTH_MASK, DISABLE_DEPTH_SORT, DISABLE_DEPTH_TEST, DISABLE_KEY_REPEAT, DISABLE_NATIVE_FONTS, DISABLE_OPENGL_ERRORS, DISABLE_OPTIMIZED_STROKE, DISABLE_STROKE_PERSPECTIVE, DISABLE_STROKE_PURE, DISABLE_TEXTURE_MIPMAPS, DODGE, DOWN, DXF, ELLIPSE, ENABLE_ASYNC_SAVEFRAME, ENABLE_BUFFER_READING, ENABLE_DEPTH_MASK, ENABLE_DEPTH_SORT, ENABLE_DEPTH_TEST, ENABLE_KEY_REPEAT, ENABLE_NATIVE_FONTS, ENABLE_OPENGL_ERRORS, ENABLE_OPTIMIZED_STROKE, ENABLE_STROKE_PERSPECTIVE, ENABLE_STROKE_PURE, ENABLE_TEXTURE_MIPMAPS, ENTER, EPSILON, ERODE, ESC, EXCLUSION, FX2D, GIF, GRAY, GROUP, HALF_PI, HAND, HARD_LIGHT, HINT_COUNT, HSB, IMAGE, INVERT, JAVA2D, JPEG, LANDSCAPE, LEFT, LIGHTEST, LINE, LINE_LOOP, LINE_STRIP, LINES, LINUX, MACOS, MACOSX, MAX_FLOAT, MAX_INT, MIN_FLOAT, MIN_INT, MITER, MODEL, MODELVIEW, MOVE, MULTIPLY, NORMAL, OPAQUE, OPEN, OPENGL, ORTHOGRAPHIC, OTHER, OVERLAY, P2D, P3D, PATH, PDF, PERSPECTIVE, PI, PIE, platformNames, POINT, POINTS, POLYGON, PORTRAIT, POSTERIZE, PROBLEM, PROJECT, PROJECTION, QUAD, QUAD_BEZIER_VERTEX, QUAD_STRIP, QUADRATIC_VERTEX, QUADS, QUARTER_PI, RAD_TO_DEG, RADIUS, RECT, REPEAT, REPLACE, RETURN, RGB, RIGHT, ROUND, SCREEN, SHAPE, SHIFT, SOFT_LIGHT, SPAN, SPHERE, SPOT, SQUARE, SUBTRACT, SVG, TAB, TARGA, TAU, TEXT, THIRD_PI, THRESHOLD, TIFF, TOP, TRIANGLE, TRIANGLE_FAN, TRIANGLE_STRIP, TRIANGLES, TWO_PI, UP, VERTEX, WAIT, WHITESPACE, WINDOWS, X, Y, Z`

- ## Constructor Summary

  Constructors
  Constructor
  Description
  `LoadAudioToImage()`

- ## Method Summary

  Modifier and Type
  Method
  Description
  `int[]`
  `adjustGamma(int[] source)`
  `void`
  `applyColor(int[] colorSource, int[] graySource, int[] lut)`
  `void`
  `audioMouseClick(int x, int y)`
  Respond to a mouse click with an audio event.
  `int`
  `calcSampleLen(int durationMS)`
  `void`
  `chooseFile()`
  `int`
  `clipToHeight(int y)`
  `int`
  `clipToWidth(int x)`
  `void`
  `draw()`
  `void`
  `fileSelected(File selectedFile)`
  `double`
  `gauss(double mean, double variance)`
  Returns a Gaussian variable using a Java library call to `Random.nextGaussian`.
  `int[]`
  `getColors()`
  `int[]`
  `getHistoBounds(int[] source)`
  `void`
  `hightlightSample(int pos, int length)`
  `int[]`
  `histoStretch(int[] source, int low, int high)`
  `void`
  `initAudio()`
  `void`
  `initMapper()`
  `void`
  `keyPressed()`
  `void`
  `loadAudioFile(File audioFile)`
  Attempts to load audio data from a selected file into playBuffer, then calls writeAudioToImage() to transcode audio data and write it to mapImage
  `static void`
  `main(String[] args)`
  `void`
  `mouseClicked()`
  `int`
  `playSample(int samplePos, int sampleCount, float amplitude)`
  Plays an audio sample with PASamplerInstrument and default ADSR.
  `void`
  `setGamma(float gamma)`
  `void`
  `settings()`
  `void`
  `setup()`
  `void`
  `showHelp()`
  `void`
  `shuffle(int[] intArray)`
  Shuffles an array of integers into random order.
  `void`
  `writeAudioToImage(float[] sig, PixelAudioMapper mapper, processing.core.PImage img, PixelAudioMapper.ChannelNames chan)`
  Transcodes audio data in sig[] and writes it to color channel chan of mapImage using the lookup tables in mapper to redirect indexing.
  `void`
  `writeImageToAudio()`

  ### Methods inherited from class processing.core.PApplet

  `abs, abs, acos, alpha, ambient, ambient, ambient, ambientLight, ambientLight, append, append, append, append, append, append, applyMatrix, applyMatrix, applyMatrix, applyMatrix, applyMatrix, arc, arc, arraycopy, arraycopy, arraycopy, arrayCopy, arrayCopy, arrayCopy, asin, atan, atan2, attrib, attrib, attrib, attribColor, attribNormal, attribPosition, background, background, background, background, background, background, background, beginCamera, beginContour, beginPGL, beginRaw, beginRaw, beginRecord, beginRecord, beginShape, beginShape, bezier, bezier, bezierDetail, bezierPoint, bezierTangent, bezierVertex, bezierVertex, binary, binary, binary, binary, blend, blend, blendColor, blendMode, blue, box, box, brightness, calcSketchPath, camera, camera, ceil, checkAlpha, checkExtension, choice, choice, circle, clear, clip, color, color, color, color, color, color, color, color, colorMode, colorMode, colorMode, colorMode, concat, concat, concat, concat, concat, concat, concat, constrain, constrain, copy, copy, copy, cos, createFont, createFont, createFont, createGraphics, createGraphics, createGraphics, createImage, createInput, createInput, createInputRaw, createOutput, createOutput, createPath, createPath, createPrimaryGraphics, createReader, createReader, createReader, createShape, createShape, createShape, createWriter, createWriter, createWriter, cursor, cursor, cursor, cursor, curve, curve, curveDetail, curvePoint, curveTangent, curveTightness, curveVertex, curveVertex, dataFile, dataPath, day, debug, degrees, delay, dequeueEvents, desktopFile, desktopPath, die, die, directionalLight, displayDensity, displayDensity, dispose, dist, dist, edge, ellipse, ellipseMode, emissive, emissive, emissive, endCamera, endContour, endPGL, endRaw, endRecord, endShape, endShape, exec, exec, exit, exitActual, exitCalled, exp, expand, expand, expand, expand, expand, expand, expand, expand, expand, expand, expand, expand, expand, expand, expand, expand, expand, expand, fill, fill, fill, fill, fill, fill, filter, filter, filter, floor, flush, focusGained, focusLost, frameMoved, frameRate, frustum, fullScreen, fullScreen, fullScreen, fullScreen, get, get, get, getExtension, getGraphics, getMatrix, getMatrix, getMatrix, getSurface, green, handleDraw, handleKeyEvent, handleMethods, handleMouseEvent, hex, hex, hex, hex, hideMenuBar, hint, hour, hue, image, image, image, imageMode, initSurface, insertFrame, isLooping, join, join, keyPressed, keyReleased, keyReleased, keyTyped, keyTyped, launch, lerp, lerpColor, lerpColor, lightFalloff, lights, lightSpecular, line, line, link, listFiles, listFiles, listPaths, loadBytes, loadBytes, loadBytes, loadFont, loadImage, loadImage, loadJSONArray, loadJSONArray, loadJSONObject, loadJSONObject, loadPixels, loadShader, loadShader, loadShape, loadShape, loadStrings, loadStrings, loadStrings, loadStrings, loadTable, loadTable, loadXML, loadXML, log, loop, mag, mag, main, main, main, makeGraphics, map, mask, match, matchAll, max, max, max, max, max, max, method, millis, min, min, min, min, min, min, minute, modelX, modelY, modelZ, month, mouseClicked, mouseDragged, mouseDragged, mouseEntered, mouseEntered, mouseExited, mouseExited, mouseMoved, mouseMoved, mousePressed, mousePressed, mouseReleased, mouseReleased, mouseWheel, mouseWheel, nf, nf, nf, nf, nf, nf, nfc, nfc, nfc, nfc, nfp, nfp, nfp, nfp, nfs, nfs, nfs, nfs, noClip, noCursor, noFill, noise, noise, noise, noiseDetail, noiseDetail, noiseSeed, noLights, noLoop, norm, normal, noSmooth, noStroke, noTexture, noTint, orientation, ortho, ortho, ortho, parseBoolean, parseBoolean, parseBoolean, parseBoolean, parseByte, parseByte, parseByte, parseByte, parseByte, parseByte, parseByte, parseByte, parseChar, parseChar, parseChar, parseChar, parseFloat, parseFloat, parseFloat, parseFloat, parseFloat, parseFloat, parseFloat, parseInt, parseInt, parseInt, parseInt, parseInt, parseInt, parseInt, parseInt, parseInt, parseInt, parseInt, parseInt, parseJSONArray, parseJSONObject, parseXML, parseXML, pause, perspective, perspective, pixelDensity, point, point, pointLight, pop, popMatrix, popStyle, postEvent, postWindowMoved, postWindowResized, pow, print, print, print, print, print, print, print, print, print, printArray, printCamera, println, println, println, println, println, println, println, println, println, println, println, printMatrix, printProjection, printStackTrace, push, pushMatrix, pushStyle, quad, quadraticVertex, quadraticVertex, radians, random, random, randomGaussian, randomSeed, rect, rect, rect, rectMode, red, redraw, registerMethod, requestImage, requestImage, resetMatrix, resetShader, resetShader, resume, reverse, reverse, reverse, reverse, reverse, reverse, reverse, rotate, rotate, rotateX, rotateY, rotateZ, round, runSketch, runSketch, runSketch, saturation, save, saveBytes, saveBytes, saveBytes, saveFile, saveFrame, saveFrame, saveJSONArray, saveJSONArray, saveJSONObject, saveJSONObject, savePath, saveStream, saveStream, saveStream, saveStream, saveStream, saveStrings, saveStrings, saveStrings, saveTable, saveTable, saveXML, saveXML, scale, scale, scale, screenX, screenX, screenY, screenY, screenZ, second, selectCallback, selectFolder, selectFolder, selectFolder, selectInput, selectInput, selectInput, selectOutput, selectOutput, selectOutput, set, set, setMatrix, setMatrix, setMatrix, setSize, shader, shader, shape, shape, shape, shapeMode, shearX, shearY, shell, shininess, shorten, shorten, shorten, shorten, shorten, shorten, shorten, showDepthWarning, showDepthWarningXYZ, showMethodWarning, showMissingWarning, showSurface, showVariationWarning, sin, size, size, size, sketchDisplay, sketchFile, sketchFullScreen, sketchHeight, sketchOutputPath, sketchOutputStream, sketchPath, sketchPath, sketchPixelDensity, sketchRenderer, sketchSmooth, sketchWidth, sketchWindowColor, smooth, smooth, sort, sort, sort, sort, sort, sort, sort, sort, sort, sort, specular, specular, specular, sphere, sphereDetail, sphereDetail, splice, splice, splice, splice, splice, splice, splice, splice, splice, splice, splice, splice, splice, split, split, splitTokens, splitTokens, spotLight, sq, sqrt, square, start, startSurface, stop, str, str, str, str, str, str, str, str, str, str, stroke, stroke, stroke, stroke, stroke, stroke, strokeCap, strokeJoin, strokeWeight, style, subset, subset, subset, subset, subset, subset, subset, subset, subset, subset, subset, subset, subset, subset, subset, subset, subset, subset, tan, text, text, text, text, text, text, text, text, text, text, text, textAlign, textAlign, textAscent, textDescent, textFont, textFont, textLeading, textMode, textSize, texture, textureMode, textureWrap, textWidth, textWidth, textWidth, thread, tint, tint, tint, tint, tint, tint, translate, translate, triangle, trim, trim, unbinary, unhex, unregisterMethod, updatePixels, updatePixels, urlDecode, urlEncode, vertex, vertex, vertex, vertex, vertex, windowMove, windowMoved, windowRatio, windowResizable, windowResize, windowResized, windowTitle, year`

  ### Methods inherited from class java.lang.Object

  `clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

- ## Field Details

  - ### pixelaudio

    PixelAudio pixelaudio
    PixelAudio library

  - ### hGen

    HilbertGen hGen

  - ### mapper

    PixelAudioMapper mapper

  - ### mapSize

    int mapSize

  - ### mapImage

    processing.core.PImage mapImage

  - ### colors

    int[] colors

  - ### chan

    PixelAudioMapper.ChannelNames chan

  - ### minim

    ddf.minim.Minim minim
    Minim audio library

  - ### audioOut

    ddf.minim.AudioOutput audioOut

  - ### playBuffer

    ddf.minim.MultiChannelBuffer playBuffer

  - ### sampleRate

    float sampleRate

  - ### audioSignal

    float[] audioSignal

  - ### rgbSignal

    int[] rgbSignal

  - ### audioLength

    int audioLength

  - ### duration

    int duration

  - ### synth

    PASamplerInstrument synth

  - ### adsr

    ADSRParams adsr

  - ### maxAmplitude

    float maxAmplitude

  - ### attackTime

    float attackTime

  - ### decayTime

    float decayTime

  - ### sustainLevel

    float sustainLevel

  - ### releaseTime

    float releaseTime

  - ### audioFile

    File audioFile

  - ### audioFilePath

    String audioFilePath

  - ### audioFileName

    String audioFileName

  - ### audioFileTag

    String audioFileTag

  - ### rando

    Random rando

  - ### roig

    int roig

  - ### groc

    int groc

  - ### blau

    int blau

  - ### vert

    int vert

  - ### taronja

    int taronja

  - ### violet

    int violet

  - ### randColors

    public int[] randColors

  - ### pixelPos

    int pixelPos

  - ### samplePos

    int samplePos

  - ### blendAlpha

    int blendAlpha

  - ### histoHigh

    int histoHigh

  - ### histoLow

    int histoLow

  - ### gammaLighter

    float gammaLighter

  - ### gammaDarker

    float gammaDarker

  - ### gammaTable

    int[] gammaTable

- ## Constructor Details

  - ### LoadAudioToImage

    public LoadAudioToImage()

- ## Method Details

  - ### main

    public static void main(String[] args)

  - ### settings

    public void settings()
    Overrides:
      `settings` in class `processing.core.PApplet`

  - ### setup

    public void setup()
    Overrides:
      `setup` in class `processing.core.PApplet`

  - ### initMapper

    public void initMapper()

  - ### initAudio

    public void initAudio()

  - ### getColors

    public int[] getColors()

  - ### draw

    public void draw()
    Overrides:
      `draw` in class `processing.core.PApplet`

  - ### keyPressed

    public void keyPressed()
    Overrides:
      `keyPressed` in class `processing.core.PApplet`

  - ### showHelp

    public void showHelp()

  - ### applyColor

    public void applyColor(int[] colorSource, int[] graySource, int[] lut)

  - ### mouseClicked

    public void mouseClicked()
    Overrides:
      `mouseClicked` in class `processing.core.PApplet`

  - ### audioMouseClick

    public void audioMouseClick(int x, int y)
    Respond to a mouse click with an audio event.

  - ### clipToWidth

    public int clipToWidth(int x)
    Parameters:
      `x` - a value to constrain to the current window width
    Returns:
      the constrained value

  - ### clipToHeight

    public int clipToHeight(int y)
    Parameters:
      `y` - a value to constrain to the current window height
    Returns:
      the constrained value

  - ### calcSampleLen

    public int calcSampleLen(int durationMS)
    Returns:
      a length in samples with some Gaussian variation

  - ### chooseFile

    public void chooseFile()

  - ### fileSelected

    public void fileSelected(File selectedFile)

  - ### loadAudioFile

    public void loadAudioFile(File audioFile)
    Attempts to load audio data from a selected file into playBuffer, then calls writeAudioToImage() to transcode audio data and write it to mapImage
    Parameters:
      `audioFile` - an audio file

  - ### writeAudioToImage

    public void writeAudioToImage(float[] sig, PixelAudioMapper mapper, processing.core.PImage img, PixelAudioMapper.ChannelNames chan)
    Transcodes audio data in sig[] and writes it to color channel chan of mapImage using the lookup tables in mapper to redirect indexing. Calls mapper.mapSigToImg(), which will throw an IllegalArgumentException if sig.length != img.pixels.length or sig.length != mapper.getSize().
    Parameters:
      `sig` - an source array of float, should be audio data in the range [-1.0, 1.0]
      `mapper` - a PixelAudioMapper
      `img` - a target PImage, modified by audio data in sig
      `chan` - a color channel

  - ### playSample

    public int playSample(int samplePos, int sampleCount, float amplitude)
    Plays an audio sample with PASamplerInstrument and default ADSR.
    Parameters:
      `samplePos` - position of the sample in the audio buffer
      `sampleCount` - number of samples to play
      `amplitude` - amplitude of the samples on playback
    Returns:
      the calculated sample length in samples

  - ### hightlightSample

    public void hightlightSample(int pos, int length)

  - ### writeImageToAudio

    public void writeImageToAudio()

  - ### gauss

    public double gauss(double mean, double variance)
    Returns a Gaussian variable using a Java library call to `Random.nextGaussian`.
    Parameters:
      `mean` -
      `variance` -
    Returns:
      a Gaussian-distributed random number with mean `mean` and variance `variance`

  - ### shuffle

    public void shuffle(int[] intArray)
    Shuffles an array of integers into random order. Implements Richard Durstenfeld's version of the Fisher-Yates algorithm, popularized by Donald Knuth. see http://en.wikipedia.org/wiki/Fisher-Yates_shuffle
    Parameters:
      `intArray` - an array of `int`s, changed on exit

  - ### getHistoBounds

    public int[] getHistoBounds(int[] source)

  - ### histoStretch

    public int[] histoStretch(int[] source, int low, int high)

  - ### setGamma

    public void setGamma(float gamma)

  - ### adjustGamma

    public int[] adjustGamma(int[] source)